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ABSTRACT 

The challenge in survey methodology is to conduct 
studies as effectively and efficiently as possible. The University of 
Newcastle upon Tyne (England) has taken advantage 'of information 
technology (IT) developments to improve the conduct of in-house 
surveys. A surveying mechanism tied to the library OPAC (online 
public access catalog) , accessible by users both inside and outside 
the library, was devised to dispense with the pencil and paper 
approach of surveying users and provide responses instantaneously in 
machine readable form. The library implemented an electronic 
surveying system that allowed: determination of respondent identity; 
screen layout to be designed by the library; comparison of 
composition of sample with whole user population; automatic analysis 
of multiple choice responses; opportunity for users to quit the 
survey at any time to return to the' OPAC; analysis of number of users 
quitting at each stage of the survey; and logging of users’ locations 
when completing the survey. The first test of the electronic survey 
system achieved a response rate of less than 107., with one-third of 
responses received from OPAC users outside the library. Low survey 
participation may be due to: (l) the survey period falling 
immediately before the examination period when students may have 
other time pressures; (2) people may not have realized the survey 
consisted of only five questions; and (3) patrons use the OPAC for a 
specific purpose, so when given the option to quit the survey to 
return to the OPAC, they do so quickly. An appendix shows sample 
survey screens. (SWC) 
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‘The right in-house study of the right issues at the 
right time is a valuable managerial resource for 
library administrators’ (Rodger, 1989). 

Many delegates at this conference will agree with 
these views, and a glance through any issue of 
Library and information science abstracts or 
Library literature will reveal many reported studies 
carried out by library managers. Much has been 
written on survey methodology; the challenge is to 
conduct such studies as effectively and efficiently as 
possible. This paper describes how the University of 
Newcastle upon Tyne has taken advantage of IT 
developments to improve the conduct of in-house 
surveys. 

The University of Newcastle upon Tyne Library 
operates from three sites - the Robinson Library, 
which is the main library, and divisional libraries 
for Medicine and Dentistry and for Law. Together, 
these libraries serve a population of over 12,000 
predominantly full-time students as well as several 
thousand academics, researchers and other staff and 
external borrowers. Reciprocal access arrangements 
with other local universities also exist 
In recent years, considerable efforts have been 
devoted to carrying out surveys to aid management 
decision-making, to inform objective setting and to 
review performance. In 1987, a university working 
party was established to review all academic ser- 
vices including the library. One particular question 
asked of the library was to provide: 

‘An indication of the Library’s strategy in achiev- 
ing a balance between quantity of service offered 
and quality of service offered’ (McDonald, 1990). 

The response to that question was informed by the 
range of strategies adopted by the library to assess 
the quality and quantity of services; including an 
evaluation of user reactions, complaints, sugges- 
tions and by conducting surveys. The ability to 
respond to the review confirmed the importance of 
in-house surveys and work in that area has been 
developed in recent years. 

Surveys have been conducted, for example, into 
the pattern of library use of Sundays, to inform the 
selection of strategy to reduce issue desk queues, to 



evaluate the student induction programme, to survey 
attitudes to reading lists and to examine cross-use of 
library services between local university students. 

However, these surveys were labour intensive. 
Questionnaires had to be distributed and collected, 
the results collated and tabulated. Interviews had to 
be conducted and the results written up. Further, 
people who came into the library had traditionally 
been surveyed, and it was desirable to try to reach 
people elsewhere on campus. 

An approach was sought where surveys could be 
conducted using computers. Newcastle University 
Library has a long history of innovation in the use 
of information technology and a computer applica- 
tion to improve surveys was seen as a natural devel- 
opment. Such a move would dispense with the pen- 
cil and paper approach and provide responses 
instantaneously in machine readable form. It was 
decided to develop a surveying mechanism which 
could be tied to the library OPAC, as many people, 
both inside and outside the library, are regular 
OPAC users. 

What was envisaged was a facility where a sam- 
ple of OPAC users would be presented with a sur- 
vey screen when first accessing the OPAC. A speci- 
fication was submitted to Geac, suppliers of the 
Advance library automated system used by 
Newcastle University, to develop an electronic sur- 
veying facility. 

The specification set out several requirements: 

- An ability to determine the respondent’s identity, 
from name or borrower number 

- Facility for the library to design the screen layout 

- Ability to compare composition of sample with 
user population as a whole 

- Automatic analysis of multiple choice responses 

- Facility for users to quit survey at any time to 
return to the OPAC 

- Analysis of the number of users quitting at each 
stage of the survey 

- Logging of users’ locations when completing the 
survey. 
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The Electronic Surveying Package 

F rom the above specification Geac developed 
software to incorporate into the Advance OPAC. 
Three aspects of the surveying package are consid- 
ered below: creating surveys, the user interface and 
survey analysis. 



CREATING SURVEYS 

Surveys are created within the Advance menu struc- 
ture used by library staff. The first step is to record 
a description of the survey and to set any start and 
finish dates. These dates are optional, but do allow 
surveys to be created before they are required and to 
come into use on the appropriate date. A ‘random 
factor’ is then selected - this determines the fre- 
quency with which the survey is offered to OPAC 
users. Whenever a valid access is made to the 
OPAC a counter is increased by 1 and then divided 
by the random factor. Whenever the remainder of 
this calculation is zero the survey is displayed. 
Therefore, if the survey was to be administered to 
every user, the factor would be set at one; for every 
tenth user the factor would be ten. 

The system allows acceptable users’ locations to 
be pre-determined. For example, if OPAC users in 
the Robinson Library only were to be surveyed, this 
could be requested. 

After these general points have been completed, 
the questionnaire can be created. 

The first element of the questionnaire is an open- 
ing screen which sets out the nature and purpose of 
the survey, and can be used, for example, to thank 
users for their help and explain why the survey is 
being conducted. The welcome screen is entirely 
free text. The second element is a prompt for the 
user’s identity. This is a divergence from practice in 
many paper-based questionnaires where confiden- 
tiality is assured, but has been adopted to allow a 
comparison to be made between sample size and 
user population, and to minimise opportunities for 
anyone to complete the questionnaire more than 
once. 

User identities can be entered in a number of 
ways: 

- By borrower number - the system checks its 
validity and prompts for a new number if neces- 
sary 

- By name - must be alphabetic and have text 
before and after a comma eg. Smith, John 

Whilst it is impossible to prevent silly names being 
entered, every effort is made to restrain practical 
jokers! 
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These two elements (Introductory screen and 
User ID) must be completed before any questions 
can be entered. 

The electronic surveying package permits a num- 
ber of question types to be established: 

- Free-text where the responder can enter anything 
(eg. What single improvement would you make 
to the library?) 

- Questions with Yes/No responses 

(eg. Are our opening hours adequate for your 
needs?) 

- Numeric questions 

(eg. How many books do you have on loan at 
present?) 

- Multiple-choice 

(eg. Which of the following extensions to open- 
ing hours would you prefer? 

a. Saturday evening 

b. Sunday morning 

c. Sunday evening) 

- Dates 

(eg. When did you last visit the library?) 

THE USER INTERFACE 

Whenever a user begins a new OPAC search the 
system retrieves a list of existing surveys - it is pos- 
sible to hold a range of surveys at any one time. The 
system will check the various elements mentioned 
above: start and finish dates and users’ location. 
Assuming that all of these restrictions are complied 
with, the system determines whether or not the 
questionnaire should be displayed using the fre- 
quency counter. 

The user prompts at the foot of each screen are 
presented and used in the same way as those with 
which OPAC users are familiar. There are two 
prompts which appear on every screen: quit and 
help. Quit is self-explanatory but nonetheless 
important. People using the OPAC generally do so 
with the intention of consulting the library’s cata- 
logue and not to complete a questionnaire, and it is 
unlikely to foster good relations if people are forced 
to complete the questionnaire before they can con- 
duct their search. Help on completing the survey is 
available at any stage from screens which explain to 
users how to move about the survey and change 
answers if required. However, it is not possible to 
amend the user ID response. 

On completion, the survey response is filed, and 
identified by either user name of borrower number. 
This is important, because if someone is presented 
with a survey and enters an identity which is already 
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held on file, they are informed that they have 
already completed the questionnaire, and returned 
to the OPAC search screen. 

SURVEY ANALYSIS 

Each completed survey response is allocated partic- 
ular data about the date and time of completion, the 
user’s ID and the location at which the response was 
prepared. At present, the software generates three 
reports to aid survey analysis. 

The first is the summary of survey control data 
which reports: 

- The survey’s dates 

- The number of times the questionnaire was pre- 
sented 

- The number of times the questionnaire was 
completed 

- The number of users who did not complete the 
survey - and the number of times quit at each 
question 

- Number of users identified by borrower no. 

- Number of users identified by name 

- Number of respondents identified as non-mem- 
bers. 

The second report offers a summary analysis of 
responses: 

- For multiple choice questions, total numbers are 
shown for each possible selection 

- For free text questions, each response is listed. 
Thirdly, the summary of survey respondents: 

- Provides a response by response analysis show- 
ing respondent’s name or borrower number and 
the time, date and location of completion. It 
then sets out the user’s response to each ques- 
tion. 

The Results to Date 

T he first live survey was carried out between 24 
April and 5 May 1995, and looked at two 
themes: opening hours and a general study of 
desired changes or additions to the library’s ser- 
vices. The questionnaire contained both multiple 
choice and free text questions. Copies of the survey 
screens may be found in Appendix 1. 

The questionnaire was presented 1,639 times; 
from which 150 completed responses were received. 
The majority of non-respondents quit before enter- 
ing their name or borrower number. One-third of 
responses were received from OPAC users outside 
the library. 



First Impressions 

DRAWBACKS 

There were a high number of non-respondents. The 
survey was presented over 1600 times, but only 150 
usable responses were gathered. A completion rate 
of less than 10% would be regarded as a disaster for 
many traditional questionnaires. 

The poor response rate can have several explana- 
tions: 

- The survey was carried out at the start of the 
final term, just before the examinations period, 
when people were likely to have other pressures 
on their time. 

- People may not have realised that the question- 
naire consisted of only five questions, and 
didn’t want to embark upon an unknown exer- 
cise. This is something which we will remedy in 
the introduction screen of future surveys. 

- People use the OPAC for a specific purpose - to 
search for catalogue records - and when given 
an easy option to quit and return to OPAC use, 
chose to do so quickly. 

After a day or so it was realised that the sample fre- 
quency would have to be shortened to gather suffi- 
cient responses. 

The second problem was the disproportionate 
number of network accesses, from people using the 
OPAC in their office or laboratory. This could clear- 
ly skew the results in favour of academics and 
researchers who are the principal users of network 
access to the OPAC. It would be important to com- 
pare the sample population to the entire user com- 
munity. 

There are a number of possible explanations for 
this problem. People on campus perhaps have more 
time and are happier to spend a minute or two com- 
pleting a questionnaire. Perhaps ‘senior’ users have 
a longer term perspective on library use. There is 
one system-related explanation for this observation 
- users logging in from the network enter the OPAC 
through the menu structure. In the library, users 
often walk away from the OPAC mid-search. The 
next user may then take over without returning to 
the home page which triggers the survey process. In 
this way, new OPAC users may be missed. 

For the first survey, considerable time was 
required to set up the screens and test the survey 
package. It is to be hoped that future surveys can be 
created rather more quickly. 
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ADVANTAGES 

Although the creation of the questionnaire was time 
consuming this was more than offset by the time 
saved by the automatic analysis of responses. 

Surveying user opinion always seems to generate 
positive publicity, as a demonstration of commit- 
ment to reviewing and improving services. 

Future Developments 

I t is intended to continue surveying, probably once 
a term. Any reduction or increase in the problems 
outlined above will be monitored closely, and steps 
will be taken to reduce or remove their impact 
At the same time, a new package - Libra - will be 
tested. Libra is a PC-based qualitative survey pack- 
age designed specifically for use in libraries. It is 
believed that Geac surveying and Libra will com- 
plement each other nicely, and may compensate for 
any respective sample bias. Libra will gather a more 
representative sample of users, whilst Geac will 
continue to reach people outside the library build- 
ing. 

Further guidance on assessing user satisfaction 
with library services will be issued by the Higher 
Education Funding Councils in due course. It is to 
be hoped that whatever framework is finally pro- 
posed, IT applications may be used to gather at least 
some of the required data. 

Traditional surveys will not disappear; there is a 
continuing need to survey services and facilities 
which have no direct link to OPAC users. However, 
developments such as those described in this paper 
make a valuable contribution to the gathering and 
analysis of valuable user opinion. 



References 

McDonald, A. C. (1990) ‘Strategies for achieving a 
balance between the quality and quantity of service 
in an academic library’. In Quality assurance and 
customer care, edited by Maggie Ashcroft and 
Alex Wilson. Stamford: CPI. p.26 

Rodger, Eleanor Jo (1989) ‘The right study of the right 
issues at the right time: conducting in-house studies 
in public libraries’. Library trends 38, Fall, p.313 



6 




196 



Keith Webster: Use of IT in Library Evaluation 



Appendix 1 

Sample survey screens 



31 AUG 1995 Geac ADVANCE Library System (5.4.4) DEFAULT NU 
14:02 Newcastle University Library 

QUESTIONNAIRE 
Page 1 of 3 

LIBRARY OPENING HOURS 

The University Library aims to provide the highest standards of service 
for all users. As an integral part of that process the Library will ask 
the opinion of its users on the services offered and, within available 
resources, will respond positively to the information received. Library 
users should expect a courteous service at all times, and comments on the 
service received and suggestions for improvement are always welcome. 

In order to help us improve our services, please answer the five 
questions below. You may enter “Q” or “q” at any time to leave the 
questionnaire and return to the catalogue. 

Please enter the number from your Library card (with no spaces and letters 
in upper case e.g. U 13040156) or your name (in the format Last name, First 
name e.g. Smith, Jane) 

Q to Quit, *?’ for help on completing the survey 



31 AUG 1995 Geac ADVANCE Library System (5.4.4) DEFAULT NU 
14:02 Newcastle University Library 

QUESTIONNAIRE 
Page 2 of 3 

1. If opening hours were to be extended (from 9am-9pm Mon-Fri, 9-4.30pm 
Sat, ll-4.30pm Sun (Robinson Library only)), when would you like it to be? 
Please indicate your choices in order of preference : 

A. 9.00 - 10.00 pm weekday evenings 

B. 4.30 - 7.00 pm Sundays (Robinson) 

C. 9.00 - 11.00 am Sundays (Robinson) 

D. 4.30 - 7.00 pm Saturdays 

2. If you would prefer the Library to be open at some other time, please 
give details below : 




Q to Quit, *?’ for help on completing the survey 
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31 AUG 1995 Geac ADVANCE Library System (5.4.4) DEFAULT NU 
14:02 Newcastle University Library 

QUESTIONNAIRE 
Page 3 of 3 

3. In order to extend opening hours as indicated above, when would you 
prefer the Library to close? Please indicate your choices in order of 
preference : 

A. 5.00 - 10.00 pm Fridays 

B. 9.00 - 12 noon Saturdays 

C. No further closing 

4. If you would prefer the Library to close at some other time, please 
give details below : 

5. Which single change or addition to the University Library service 
would you like to see implemented ? 

Thanks for your assistance 
Q to Quit, *?’ for help on completing the survey 
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